Phase: Managing Test Process

Testing software (information system, package implementation or embedded software) is usually organised with a number of test levels. Each test level has a specific aim, e.g. establishing the correct functioning of a component or a system’s adequate quality for production. When the test manager of each test level, in consultation with his direct clients, decides what will be tested, chances are that in the total picture of testing, certain matters will be tested twice unnecessarily, or that to the contrary specific issues are forgotten. The method should be vice versa: based on the total overview a test manager, in consultation with the client and other stakeholders, makes a division as to which test level tests what and when (and with what intensity), fitting within the implemented development or maintenance approach. The aim is to detect the most important defects as early and economically as possible. This agreement is laid down in the so-called master test plan ( MTP). This plan constitutes the basis for the detailed test plans for the separate test levels. In addition to the content-based alignment, there are other reasons to try and gear activities to one another: ensuring uniformity in processes (e.g. the defect procedure and testware management), availability and management of the test environment and tools, and optimal division of resources (both people and means) across the test levels.

This phase is about the management of the total test process based on the BDTM philosophy, and is therefore not limited to creating the master test plan. Coordination and adjustment is vital when executing the test plans. Delayed deliveries, disappointing quality, or a lack of time or resources are more of a rule than the exception in IT. The test manager of a test level must then adjust the planned approach. Coordinating and managing the total test process across the test levels must ensure that, despite individual adjustments in various test levels, a coherent overall approach for testing continues to be implemented. Adjustment in a test level may result in inefficiency in other test levels, e.g. because the test object has insufficient quality when completing the first test level. The test manager must notify others and propose or take measures in consultation with the client and project management. The socalled Deming wheel [Deming, 1992] can be distinguished in the management and improvement mechanism used to this end: Plan something, Do it, Check the result, and Act if necessary. See the figure below.

In some cases, in particular for maintenance and outsourcing, large parts of the master test plan are created with reuse in mind. This is the last variant described in this chapter, the Generic Test Agreements (GTAs).

Figure 1: Management of master test plan and test levels